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{D e script i on} [Substitute Specification:] 



{M e thod and arrang e m e nt for d e t e rm i ning a mov e m e nt which und e rli e s a d i g i t i z e d 



tmage}[- - METHOD AND SYSTEM FOR DETERMINING ] 



fFh^[MOVEMENT UNDERLYING A DIGITIZED IMAGE 



BACKGROUND OF THE INVENTION 



Field of the Invention 

The present invention relates to digital video processing. In particular, 
the present] invention relates to the determination of {a} movement which underlies 
a digitized image. 
[Discussion of the Related Art] 

A method for determining a movement which underlies a digitized image is 
{known from [1] and [2].} [desxrcibed in, *A Noise Robust Method for 2D Shape 
Estimation of Moving Objects in Video Sequences Considering a Moving 
Camera* by R. Mech, M. Wollborn, which appeared in Workshop on Image 
Analysis for Multimedia Interactive Services, Belgium, June 1997, as well as in 
an article by S. Colonnese et al., entitled ^Adaptive Segmentation of Moving 
Object versus Background for Video Encoding* which appeared in 
Proceedings of SPIE Annual Symposium, Vol. 3164, San Diego, August 1997.] 

{In th e m e thod from [1]} [According to the Mech and Wollborn article,] a 
global relative movement between a camera and a sequence of images taken by the 
camera is determined. {Th e } [Their] method {from [1]} , which is used in the image 



stabilization of a camera, is based on a very inaccurate movement model which can 
describe only a tilting of the camera. 

This disadvantage of a substantial inaccuracy in the determination of the 
global movement is also inherent to the method {from [2] wh i ch m e thod} [presented 
by Colonnese et al., which ]is used in the segmentation of the digitized image. 

In order to achieve an improved accuracy, it is known to base the 
{d e t e rmination} [determina-tion] of a movement on a more complex movement 
model which is determined, with the aid of gradients in the digitized image, on the 
level of the pixels which are contained in the image {. I l ow e v e r, th i s} [, such as 
presented by S.S. Beauchemin, J.L. Barron in *The Computation of Optical 
Flow* ACM Computing Surveys, Vol. 27, No. 3, pages 366-433, September 
1995. However, hSs] method is complicated, and can therefore be carried out only 
with a {r e quir e ment for} substantial [amount of] computing time. 

Furthermore {[4] d i sc l os e s} [, in the article entitled "^Displacement 
Estimation by Hierarchical Blockmatching* by M. Bierlin, which appeared in 
SPIE, Vol. 1001, Visual Communications and Image Processing '88, pages 942 
- 951, 1988, presents] a method for so-called movement estimation { i n a m e thod} 
for block-based image encoding. In this method, it is assumed that a digitized image 
has pixels which are grouped in image blocks of usually 8 {*}[x] 8 pixels or 16 {*}[x] 
16 pixels. { 

}Furthermore, an image block is to be understood both as an image block of, for 
example 8 {*}[x] 8 pixels or 16 {*}[x] 16 pixels, and also a set of image blocks, for 
example a so-called macroblock, which contains 6 image blocks{(}[, of which,] 4 
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image blocks {w i th} [hold] brightness informationf;;} [and] 2 image blocks {w i th} 
[hold] color informationf)}. 

Within the framework of a sequence of temporally succeeding images, for 
each image block the following method is carried out for an image to be coded for an 
image block in the image to be coded and a temporally preceding, already coded 
image: [(1) an]f--Afr} error value of an error dimension is formed for the image block, 
for which a movement estimation is being carried out, in the temporally preceding 
image, starting from an image block which is located in the same relative position in 
the temporally preceding image, denoted below as a preceding image block, this 
being done, for example, by forming a sum over the absolute values of the 
differences of encoding information, assigned to the pixels, of the image block and 
the preceding image block. { 

}ln this connection, encoding information is to be understood as brightness 
information (luminance value) and/or color information (chrominance value), which is 
respectively assigned to a pixel[; (2) in]f7 

-4fr) a search space of prescribable size and shape about the initial position in the 
temporally preceding image, an error value of the error measure is formed in turn in 
each case in a region of the same size of an image block (preceding image block), 
displaced in each case by one or half a pixel[; (3) thisjfr 
" Th i s} results in n^ error values in a search space of size n * n pixels. That 
"displaced" preceding image block in the temporally preceding image is selected for 
which the error measure yields a minimum error value. It is assumed for this image 
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block that this preceding image block corresponds best to the image block of the 
image to be coded for which the movement estimation is carried out[; (4) ]{:) 
{" Th e } [the] result of the movement estimation is a movement vector with which the 
displacement between the image block in the image to be coded and the selected 
image block in the temporally preceding image is described[; (5) image]{7 

" I mag e } data compression in the case of the block-based image encoding is 
achieved by virtue of the fact that only the movement vector and an error signal are 
coded[; and (6) the]{7 

" Th e } movement estimation is carried out for each image block of an image. 

However, the method described in {{4}) [the Bierlin article referred to 
above,] cannot be used for a "global" movement estimation, {that} [which] is {to 
say} [the] determination of {the} movement between a camera and the scene taken 
by the camera. 

This is {ascr i b e d, i n part i cu l ar,} [due] to the heterogeneity of an image with a 
multiplicity of objects which are moving in different ways in the image. { 

}The application of the movement estimation to block-based image encoding, or 
{ el s e } to object-based image encoding { i s known from [5] and [6].} [, is discussed in 
ITU-T, International Telecommunication Union, Tele-communications Sector of 
ITU, Draft ITU-T Recommendation H.263, Video-Encoding for Low Bit-Rate 
Communication, 2nd May 1996.] 

{The^fThe present] invention is therefore based on [solving] the problem of 



{d e t e rmining} [determin-ing] and ascribing a nnovennent which underlies a digitized 
image { 

}in a simple, fast and cost effective way[, and can be used to improve the image 
segmentation method described by Colonnese et al., above.]{7 

Th e prob le m i s solv e d by m e ans of th e m e thod i n accordanc e w i th pat e nt c l a i m 1 , 
and by m e ans of th e arrang e m e nt in accordanc e w i th pat e nt c l a i m 10.} 

The method for computer-aided determination of a movement which underlies 
a digitized image [considers] {compr ise s th e follow i ng st e ps: 
-} the digitized image contains pixels which are grouped into image blocks[; a 
movement estimation is carried out for each image blocl^, as a result of which 
a movement vector is determined for each image blocic, which movement 
vector is assigned to the respective image block; movement vectors are 
selected which are assigned to an image block which is situated in a 
prescribed region of the digitized image; parameters of a movement model are 
determined from the selected movement vectors; and the movement of the 
digitized image is described by the determined movement model.Jf;;} 

{<^[The method and system for computer-aided determination of a 
movement which underlies a digitized image according to the present 
invention uses a processor which is set up in such a way that the digitized 
image contains pixels which are grouped into image blocks,] a movement 
estimation is carried out for each image block, as a result of which a movement 
vector is determined for each image block, which movement vector is assigned to 
the respective image block, { 




-}movement vectors are selected which are assigned to an image block which is 
situated in a prescribed region of the digitized image, { 

-}parameters of a movement model are determined from the selected movement 
vectors, and { 

-}the movement of the digitized image is described by the determined movement 
model. 

The [present invention provides an efficient, simple method and system, 
which can be carried out cost-effectively with a substantially reduced 
computing requirement. Furthermore, the present invention uses movement 
vectors which are determined by block-based image encoding, which itself is 
used to determine a global movement between a camera and a scene taken by 
the camera. However, when determining the movement, account is taken only 
of movement vectors ] {arrang e m e nt for comput e r-aid e d d e t e rm i nat i on of a 
mov e m e nt wh i ch und e r l i e s a d i gitiz e d image has a proc e ssor wh i ch i s s e t up i n such 
a way that th e fol l owing steps can b e carr ie d out: 

" th e d i g i t i z e d imag e conta i ns pix el s which ar e group e d i nto i mag e b l ocks, 
" a mov e m e nt e stimat i on i s carri e d out for e ach i mag e b l ock, as a r e su l t of wh i ch a 
mov e m e nt v e ctor i s d e t e rm i n e d for e ach i mag e b l ock, wh i ch mov e m e nt v e ctor i s 
ass i gn e d to the r e sp e ctiv e i mag e b l ock, 

" mov e m e nt v e ctors ar e select e d} which are assigned to {an} image {b l ock wh i ch i s} 
[blocks] situated in a prescribed region {of th e d i gitiz e d i mag e ,} [.] 



{ ' parameters o f a m o vemen t m od e l a r e de t erm i ned f r om t he selecte d 
move men t vectors, and} [SUMMARY OF THE INVENTION! 
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th e mov e m e nt of th e d i g i t i z e d i mag e is d e scr i b e d by th e d e t e rm i n e d 
mov e m e nt mod el . 

Th e m e thod provid e s an e ff i ci e nt, s i mpl e m e thod, wh i ch can th e r e for e b e carri e d out 
cost^' e ff e ctiv e ly w i th a substantia ll y le ss e r comput i ng r e qu i r e m e nt, and an 
arrang e m e nt wh i ch can th e r e for e b e i mpl e m e nt e d cost^ e ff e ct i v el y. 

Th e i nv e nt i on i s to b e s ee n c le ar l y i n that mov e m e nt v e ctors which ar e d e t e rm i n e d in 
any cas e with th e b l ock^bas e d imag e e ncod i ng ar e us e d to d e t e rmin e a globa l 
mov e m e nt b e tw ee n a cam e ra and a sc e n e tak e n by th e cam e ra. 

I l ow e v e r, wh e n d e t e rm i n i ng th e mov e m e nt account i s tak e n on l y of mov e m e nt 
v e ctors which ar e ass i gn e d to} [It is an object of the present invention to provide 
a method and system for determining movement underlying a digitized image 
wherein a prescribed region is formed by Jimage blocks which are situated {tn-a 
pr e scr i b e d r e g i on. 

Advantag e ous d e v el opm e nts of th e inv e ntion fo ll ow from th e d e p e nd e nt c l a i ms. 

I n a d e v el opm e nt of th e inv e nt i on, it i s advantag e ous that th e pr e scr i b e d r e g i on i s 
form e d by i mag e blocks wh i ch ar e s i tuat e d} at a prescribed first distance from an 
edge of the digitized image and/or at a prescribed second distance from the middle 
of the digitized image. 
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{Th i s d e v el opm e nt is bas e d on th e f i nding that} [It is another object of the 
present invention to provide a method and system for determining movement 
underlying a digitized image wherein] movement vectors of image blocks which 
are situated at the edge of the image generally specify the {actual mov e m e nt on l y 
unr el iab l y. Furth e rmor e ,} [movement reliably. 

It is a further object of the present invention to provide a method and 
system for determining movement underlying a digitized image wherein] 
zooming and rotating of a camera can be specified {on l y unr eli ab l y} [reliably] by 
movement vectors which are assigned to image blocks which are grouped in a 
region around the middle of the image. 

{ I n th i s cas e ,} [It is an additional object of the present invention to 
provide a method and system for determining movement underlying a digitized 
image wherein] the prescribed region clearly forms a "mask" in the form of a 
"perforated" rectangle inside the digitized image. 

{A furth e r d e v el opm e nt consists i n introduc i ng it e rat i ons i n d e t e rm i ning } [lt is 
yet another object of the present invention to provide a method and system for 
determining movement underlying a digitized image involving the introduction 
of iterations to determine] the movement model by modifying the "mask" after 
determining the parameters of the movement model and using this modified "mask" 
to recalculate the parameters of the movement model. 

{Th e "mask" can b e modifi e d i n this cas e , for e xamp le , by virtu e of th e fact 
that blocks whos e mov e m e nt v e ctors deviat e from thos e of th e mov e m e nt mod el , 
and th i s d e v i at i on e xc ee ds a thr e shold va l u e with r e f e r e nc e to a pr e scribab le 
d i stanc e m e asur e , ar e eli minat e d from th e prescrib e d r e g i on. 
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A furth e r r e f i n e m e nt cons i sts in} [It is yet a further object of the present invention 
to provide a method and system for determining movement underlying a 
digitized image by] forming the prescribed region by image blocks whose 
movement it was possible to estimate particularly reliably. This can be detected, for 
example, by virtue of the fact that the associated prediction error is below a 
prescribed threshold, or the variance of the prediction error in the search zone is 
above a threshold. 

{furth e rmor e ,} [It is yet an additional object of the present invention to 
provide a method and system for determining movement underlying a digitized 
image wherein] it is possible to use a "weighting mask" instead of the binary 
"mask" {d e scrib e d i n th e for e go i ng paragraphs. I n th i s cas e , i t i s not, as pr e vious l y 
d e scr i b e d, } [, using] blocks or their movement vectors which are discretely selected 
for further calculation {, but th e b l ocks or th e ir mov e m e nt v e ctors ar e w ei ght e d with 
factors. Th e s e can b e d i ff e r e nt for th e X-compon e nt and Y^compon e nt of th e 
movem e nt v e ctor. Thes e w ei ghtings f e atur e i n th e ca l culation of th e param e t e rs of 



of th e arrang e m e nt w i th th e a i d of which an i mag e i s tak e n.} [These and other 
objects and advantages of the present invention will become apparent upon 
careful review of the following detailed description of the preferred 
embodiments which is to be read in conjunction with review of the following 
drawing figures.] 

rrhe inven ti on can b e use d t o compensate a came r a m o vement or a l s o t o 




[•] 



{Th e d e t e n 



le nt can b e us e d to comp e nsat e an actua l mov e m e nt 
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compen s ate a movement of a mob il e commun i ca ti on d ev i ce wh i ch i ncludes 

t he came ra: 

An exemplarv embod i ment of t he i nven t ion i s il lus t rated in t he draw i ngs a nd 

e xpla i ned i n more deta il be l ow. 

*n the draw i n ot} TBRIEF DESCRIPTION OF THE DRAWINGS] 

Figure 1 shows a block diagram { i n which th e pr i nc i ple of th e ex e mp l ary 

e mbod i m e nt is ill ustrat e d p i ctor i a ll y;} [according to the present invention;] 
Figure 2 shows a sketch of {an arrang e m e nt with a cam e ra and an 

e ncoding unit for e ncod i ng th e i mage sequ e nc e tak e n with th e cam e ra, and an 
arrangem e nt for d e coding th e e ncod e d i mag e s e qu e nc e; } [a coding and encoding 
of an image sequence according to the present 

invention; ] 

{F i gur e 3 shows a d e ta i l e d sk e tch of th e arrangem e nt for imag e e ncod i ng and} 
[Figure 3 shows an image encoding] for global movement 

compensation [according to the present 

invention; ]{t} 

{figures 4a to c r e sp e ct i v el y show an i mag e i n which a} [Figures 4a - 4c show 
processing of an image] movement vector field {is d e t e rmined for th e imag e 
r e lat i v e to a t e mpora l ly pr e c e ding i mag e with a pr e scr i b e d r e gion (f i gur e 1a) from 
wh i ch i n e ach cas e th e mov e m e nt v e ctors ar e d e t e rm i n e d for forming param e t e rs of 
a mov e m e nt mod el , an imag e w i th a l l th e mov e m e nt v e ctors (f i gur e 1b) and an 
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i mag e w i th mov e ment v e ctors aft e r i t e rat i on of th e m e thod w i th th e pr e scr i b e d r e g i on 
ill ustrat e d i n figur e 1a (Figur e 1c);} [according to the 

present invention; and] 

Figure 5 shows a flowchart { i n which th e m e thod st e ps of th e e x e mplary 

e mbod i m e nt ar e ill ustrat e d.} [according to the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 



Figure 1 shows, in block diagram form, the principle on which the global 
movement determination is based. 

The parameters of the movement model 338 described below are 
calculated (step 103) starting from a movement vector field 101, the prescribed 
region or a weighting mask 102 and a weighting mask of reliability factors 106. 

A movement vector field 101 is understood to be a set of all the 
determined movement vectors 330 relating to an image. The movement vector 
field 101 is illustrated (402) in Figure 4b by strokes which in each case 
describe a movement vector 330 for an image block. The movement vector 
field 402 is sketched on the digitized image 400. The image 400 comprises a 
moving object 403 in the form of a person, and an image background 404.] 

Figure 2 illustrates an arrangement which comprises two computers 202, 208 
and a camera 201, image encoding, { 

)transmission of the image data and image decoding being illustrated. 
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A camera 201 is connected to a first computer 202 via a line 219. The camera 
201 transmits taken images 204 to the first computer 202. The first computer 202 
has a first processor 203, which is connected to an image store 205 via a bus 218. 
The method for image encoding is carried out with the aid of the first processor 203 
of the first computer 202. Image data 206 encoded in this way are transmitted from 
the first computer 202 via a communication link 207, preferably a line or a radio 
path, to a second computer 208. The second computer 208 includes a second 
processor 209, which is connected to an image store 211 via a bus 210. A method 
for image decoding is carried out with the aid of the second processor 209. 

Both the first computer 202 and the second computer 208 each have a 
display screen 212 and 213, respectively, on which the image data 204 are 
visualized. Input units, preferably a keyboard 214 and 215. respectively, and a 
computer mouse 216 and 217, respectively, are respectively provided for operating 
both the first computer 202 and the second computer 208. 

The image data 204, which are transmitted to the first computer 202 by the 
camera 201 via the line 219 are data in the time domain, while the data 206, which 
are transmitted via the communication link 207 to the second computer 208 by the 
first computer 202 are image data in the spectral region. { 

)The decoded image data are illustrated on a display screen 220. 

Figure 3 shows a sketch of an arrangement for carrying out a block-based 
image encoding method in accordance with the H.263 standard (see [5]). 

A video data stream which is to be encoded and has temporally succeeding 
digitized images is fed to an image encoding unit 301. The digitized images are 
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subdivided into macroblocks 302, each macroblock containing 16x16 pixels. The 
macrobiock 302 comprises 4 image blocks 303, 304, 305 and 306, each image 
block containing {8x8} [8 x 81 pixels to which luminance values (brightness values) 
are assigned. Each macroblock 302 further comprises two chrominance blocks 307 
and 308 with chrominance values (color information, color saturation) assigned to 
the pixels. 

The block of an image includes a luminance value (= brightness), a first 
chrominance value (= shade) and a second chrominance value (= color saturation). 
In this case, the luminance value, first chrominance value and second chrominance 
value are denoted as color values. 

The image blocks are fed to a transformation encoding unit 309. In differential 
image encoding, values, to be encoded, of image blocks of temporally preceding 
images are subtracted from the image blocks currently to be encoded, and only the 
differential imaging information 310 is fed to the transformation encoding unit 
(Discrete Cosine Transformation, DCT) 309. For this purpose, the current 
macroblock 302 is communicated via a connection 334 to a movement estimation 
unit 329. Spectral coefficients 31 1 are formed in the transformation encoding unit 
309 for the image blocks or differential image blocks to be encoded, and are fed to a 
quantization unit 312. 

Quantized spectral coefficients 313 are fed both to a scanning unit 314 and to 
an inverse quantization unit 315 in a return path. Entropy encoding is 
carried out on the scanned spectral coefficients 332 in an entropy encoding unit 316 
provided therefor using a scanning method, for example a zigzag scanning method. 
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The entropy-encoded spectral coefficients are transmitted as encoded image 
data 317 to a decoder via a channel, preferably a line or a radio path. 

Inverse quantization of the quantized spectral coefficients 313 is performed in 
the inverse quantization unit 315. Spectral coefficients 318 thus obtained are fed to 
an inverse transformation encoding unit 319 (Inverse Discrete Cosine 
{Transformation} [Transforma-tion], IDCT). Reconstructed encoding values (also 
differential encoding values) 320 are fed to an adder 321 in the differential image 
mode. The adder 321 also receives encoding values of an image block which result 
from a temporally preceding image after movement compensation which has already 
been carried out. Reconstructed image blocks 322 are formed with the aid of the 
adder 321 and stored in an image store 323. 

Chrominance values 324 of the reconstructed image blocks 322 are fed from 
the image store 323 to a movement compensation unit 325. Interpolation in a 
specifically provided interpolation unit 327 is performed for brightness values 326. 
The interpolation is used to preferably double the number of brightness values 
contained in the respective image block. All brightness values 328 are fed both to 
the movement compensation unit 325 and to the movement estimation unit 329. The 
movement estimation unit 329 also receives the image blocks of the particular 
macroblock (16x16 pixels) to be encoded, via the connection 334. The movement 
estimation is performed in the movement estimation unit 329 taking account of the 
interpolated brightness values ("movement estimation on a half-pixel basis"). 

The result of the movement estimation is a movement vector 330 which 
expresses a spatial displacement of the selected { 
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}macroblock from the temporally preceding image to the macroblock 302 to be 
encoded. 

Both brightness information and chrominance information relating to the 
macroblock determined by the movement estimation unit 329 are displaced by the 
movement vector 330 and subtracted from the encoding values of the macroblock 
302[,] (see data path 231). 

The way in which the movement estimation is performed is to determine for 
each image block for which a movement estimation is carried out an error E with 
respect to a zone of the same shape and size as the image block in a temporally 
preceding image, doing so, for example, in accordance with the following rule: 



n m 

E = Z Z \^i,3 - ^^ifj\ Vd G S , (1) 

i = lj = l 



- i, j denote respectively indices, 

- n, m denote, respectively, a number (n) of pixels along a first direction x, and a 
number (m) of pixels along a second direction y, which are contained in the image 
block, 

- Xjj denote respectively the encoding information which is assigned to a pixel at the 
relative position, denoted by the indices i, j, in the image block, 

- xdj j denote respectively the encoding information which is assigned to the 
respective pixel, denoted by i, j, in the zone of the temporally preceding image, 
displaced by a prescribable value d, and 
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- S denotes a searched space of prescribed shape and size in the tennporally 
preceding innage. 

{Th e ca l cu l at i on} [Calculation] of the error E is carried out for each image 
block for different displacements within the search space S. That image block in the 
temporally preceding image whose error E is minimum is selected as most similar to 
the image block for which the movement estimation is carried out. 

The result of the movement estimation is therefore yielded as the movement 
vector 330 with two movement vector components, a first movement vector 
component BV^ and a second movement vector component BVy along the first 
direction x and the second direction y: 

The movement vector 330 is assigned to the image block. 

The image encoding unit from Figure 3 therefore supplies a movement vector 
330 for all image blocks or macroimage blocks. 

The movement vectors 330 are fed to a unit 335 for selecting or weighting the 
movement vectors 330. In the unit for selecting the movement vectors 335, those 
movement vectors 330 are selected or highly weighted which are assigned to image 
blocks which are located in a prescribed region 401 (compare Figure 4a). 
Furthermore, movement vectors which have been reliably (342) estimated are 
selected or highly weighted in the unit 335. 

The selected movement vectors 336 are fed to a unit for determining the 
parameters of the movement model 337. The movement model in accordance with 
Figure 1, {wh i chis} [which is] described below, is determined from the selected 
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movement vectors in the unit for determining the parameters of the movement 
model 337. 

The determined movement model 338 is fed to a unit for compensating 339 the 
movement between the camera and the taken image. The movement is 
compensated in the unit for compensating 339 in accordance with a movement 
model described below, and so a movement-compensated image 340 which is less 
shaky is stored again, after processing in the unit for compensation 339, in the 
image store 323 in which the previously non-processed image whose movement is 
to be compensated is stored. 

{f i gur e 1 shows in th e form of a b l ock diagram th e pr i nc i pl e on which th e 
globa l mov e m e nt d e t e rm i nat i on i s bas e d. 

Th e paramet e rs of th e mov e m e nt mod el 338 d e scr i b e d below ar e ca l culat e d (st e p 
103) starting from a mov e m e nt v e ctor fi el d 101, th e pr e scr i b e d r e g i on or a w e ighting 
mask 102 and a w e ight i ng mask of r eli ab ili ty factors 10 6 . 

A mov e m e nt v e ctor f iel d 101 i s und e rstood to b e a s e t of a ll th e d e t e rm i n e d 
movem e nt v e ctors 330 r el at i ng to an i mag e . Th e mov e m e nt v e ctor fi e ld 101 is 
ill ustrat e d (402) i n f i gur e 4b by strok e s wh i ch in e ach cas e d e scr i b e a mov e m e nt 
v e ctor 330 for an i mage b l ock. Th e mov e m e nt v e ctor f i e l d 402 i s sk e tch e d on th e 
d i git i z e d imag e 400. The i mag e 400 comprises a mov i ng obj e ct 403 i n the form of a 
p e rson, and an i mag e background 404. 

}Figure 4a shows a prescribed region 401 . The prescribed region 401 specifies a 
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zone in which the image blocks must be situated so that the movement vectors 
which are assigned to these Image blocks are selected. 

The prescribed region 401 results from the fact that an edge region 405 which 
is formed by image blocks which are situated at a prescribed first distance of 406 
{froman} [from an] edge 407 of the digitized image 400 [lacuna]. Image blocks are 
therefore not taken directly into account at the edge 407 of the image 400 when 
determining the parameters of the movement model 338. Furthermore, the 
prescribed region 401 is formed by image blocks which are situated at a prescribed 
second distance 408 from the middle 409 of the digitized image 400. 

The prescribed region or the weighting mask is varied in an iterative method 
having the following steps to produce a new region of the following iteration (step 
104). 

For each image block in the prescribed region 401 , a vector difference value 
VU is respectively determined, with the aid of which the difference of the determined 
movement model 338 with the movement vector 330 which is assigned to the 
respective image block is described. The vector difference value VU is formed, for 
example, in accordance with the following rule: 

VU = {(DVX " MDVX( I (DVY - MDVY(,} [*BVx - MBVx* + *BVy - MBVy*,] 

(2) 

MBVj( and MBVy respectively denoting the components of a movement vector MBV 
calculated on the basis of the movement model. 

The determination of the model-based movement vector is explained below in 
more detail. 
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In the case of the use of a binary mask, an image block is included in the new 
region of the further iteration when the respective vector differential value VU is 
smaller than a prescribable threshold value {(:)[*.] However, if the vector differential 
value VU is greater than the threshold value {0[*1 the image block to which the 
respective movement vector is assigned is no longer taken into account in the new 
prescribed region, f 

}ln the case of the use of a weighting mask, the weighting factors of the blocks are 
specified in the reverse ratio to that of the VU thereof. 

As a result of this mode of procedure, those movement vectors which differ 
substantially from the movement vectors MBV calculated from the determined 
movement model are not taken into account, or are taken into account only slightly 
in calculating the parameters of the movement model in a further iteration. 

After the new region or the new weighting mask has been formed, the 
movement vectors are used to assign the image blocks which are not included in the 
new region, or a new set of parameters is determined for the movement model by 
making additional use of the weighting mask. 

The method described above is carried out in a prescribable number of 
iterations or until a stop criterion, such as the undershooting of a number of 
eliminated blocks in an iteration step, for example, is fulfilled. 

In this case, the new region is used in each case as the prescribed region or 
the new weighting mask in addition to the old movement vectors as input parameters 
of the next iteration. { 
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}The determination of the global movement is carried out in such a way that 
parameters of a model for the global camera movement are determined. 

A detailed derivation of the movement model is illustrated below in order to 
explain the movement model{t)[.] It is assumed that a natural, three-dimensional 
scene is being projected by the camera onto a two-dimensional plane of projection. 
A projection of a point 

is formed in accordance with the following rule: 



fx" 



= A Zn » F , 



F describing a focal length and X,Y describing coordinates of the projected point Qq 
on the image plane. 

If the camera is now moved, the projection rule is maintained in the 
coordinate system simultaneously moved synchronously with the camera, but the 
coordinates of the object points must be transformed into this coordinate system. 
Since all the camera movements can be considered as an accumulation of rotation 
and translation, the transformation of the fixed coordinate system (x, y, z) into a 

simultaneously moved coordinate system V^O^yO'^/ Qgn be formulated in 

accordance with the following rule: 
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(6) 



Starting from rule (6) a change in image caused by camera movement is 
modeled in accordance with the following rule: 



UyJ I Cp sin(<P2:) Cp cos(<^-^) - ij ' 



tx 



{(X, (Y} [*X, *Y] denoting a variation in the pixel coordinates caused in a time 
interval {ft}[*t] in the case of the described camera movement, and {(2^[%i denoting 
the angle by which the camera has been rotated about a z-axis in this time interval 
{(t}[*t]. A prescribed factor Cp denotes a change in focal length or a translation along 
the z axis. 

The system of equations represented in rule (7) is nonlinear, for which reason 
the parameters of the system of equations cannot be determined directly. 
{ 

)Consequently. a simplified movement model is used for more rapid calculation, and 
in this case the camera movement in the plane of projection is used by a movement 
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model with 6 parameters which are formed in accordance with the following rule: 



^11 ^2 

I I 

U2I ^22 J 



tx 

I 



8) 



The system of equations produced therefrom with the data of the movement 
vector field is solved by means of linear regression, the complexity corresponding to 
inversion of a symmetrical 3 {*}[x] 3 matrix. 

After determination of the parameters r",,, r'12, r'21, r'22, t'x and r'y the 
parameters of rule (7) are approximated in accordance with the following rules: 



1 = 1' 



(9) 



Cf = 



'det 
{10 



KX21 122 > 



Pz = arcsin ^ (r2i - 1^2) • 



(11) 
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The movement which underlies an image relative to a camera which takes the 
image is compensated with the use of these parameters. 

Figure 4c shows the movement vectors which are assigned to image blocks 
which are situated in the prescribed region 401. In this case, the prescribed region 
401 is varied by an iteration (step 104) with respect to the prescribed region 401 
from Figure 4a. 

The method will be illustrated once again in terms of its individual method 
steps with the aid of Figure 5{t)[-] 

After the method has started (step 501), an image block or macroimage block 
is selected (step 502). A movement vector is determined (step 503) for the selected 
image block or macroimage block, and a check is made in a further step (step 504) 
as to whether all the image blocks or macroimage blocks of the image are 
processed. 

If this is not the case, a further image block or macroimage block which has 
not yet been processed, is selected in a further step (step 505). 

If, however, all the image blocks or macroimage blocks are processed, the 
movement vectors are selected which are assigned to an image block or a 
macroimage block which are situated in the prescribed region (step 506). 

The parameters of the movement model are determined (step 507) from the 
selected movement vectors. If a further iteration is to be carried out, that is to say if 
the prescribed number of iterations has not yet been reached or the stop criterion is 
not yet fulfilled, a new region is determined in a further step (step 509), or the 
weighting mask ( 

}of the next iteration is calculated as a function of the vector differential values VU 
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(step 510). ( 



}This is followed by compensating the movement of the image by using the 
determined movement model (step 508). 

Some altematives to the exemplary embodiment illustrated above are 
explained below: 

The form of the region is fundamentally arbitrary and preferably dependent on 
prior knowledge of a scene. No use should be made in determining the movement 
model of those image regions of which it is known that these image regions differ 
clearly from the global movement. 

The region should include only movement vectors of image regions which 
have proved to be reliable on the basis of the reliability values 342 of the movement 
estimation method. 

In general, the movement estimation can be performed using any desired 
method, and is in no way limited to the principle of block matching. Thus, for 
example, movement estimation can also be performed using dynamic programming. 
Consequently, the type of movement estimation, and thus the way in which a 
movement vector is determined for an image block, are irrelevant to the [present] 
invention. 

As an alternative to the approximate determination of the parameters of the 
system of equations (7), it is possible to linearize the sine terms and cosine terms in 
rule (7). 

The following rule therefore results for small angles {f^[*J 
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Since the optimizations of the equations for {(X}[*X] and {(¥}[*¥] are not 
mutually independent, minimization is carried out with respect to the sum of the 
squares of the errors, that is to say in accordance with the following rule: 



V 



min 



(13) 



Here, {(X(, (Y(} [*X., *Y*j denote the X- and Y-components, respectively, of the 

movement vector of the image block {(}[*] at the position X^^_^j*, Y*j of the prescribed 
region V of the image. 

In accordance with equation (12), R^. R2, t^ and ty are the parameters of the 
movement model which are to be determined. 

After the optimization method has been carried out, the associated 
model-based movement vector MBV ( {(X, (Y)} [*X, *Y)] is determined on the basis of 
the determined system of equations (12) by substituting the X- and Y-components of 
the respective macroblock. 

Instead of the abovenamed regions, it is also possible to make use of 
weighting masks A,,, Ay which separately represent the reliability of the movement 
vectors, the a priori knowledge and the conclusions from the VU in the iterative 
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procedure for the X- and Y-components of the movement vectors when calculating 
the parameters of the movement model in accordance with the following 
optimization formulation: 



z 

V 



(ax^ • (^T, - RlXn + R2YT1 - tx))^ + 



min 



(14) 



A weighting mask A^, Ay for the reliability of the movement vectors (105) can 
be calculated, for example, by calculating the values {(x, (y} [*x, %] for an image block 
in the following way In the case of block matching: 



ttv = 



SADmatch 



Z 

N 



I SADt^ - SADmatchl 
x-q - Xmatch 



[151 
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ay = 



SADitiatch 



N 



I SADt^ - SADmatchI 



Yti " Ymatch 



(16) 



SAD^f*j representing the sum of the pixel differences of a block for the 
{fth)[**^^ displacement (x^^^p, y*)] of the block matching, and SAD^gj^^ representing 
the same for the best, finally selected zone (x^atch* ymatch)- N is the total number of 
search positions which have been investigated. If this value is calculated only taking 
account of the, for example, 16 best zones, the block matching can be carried out as 
a "spiral search" with the SAD of the worst of the 16 selected zones as stop criterion. 

A further possibility of calculating a weighting mask A^ = Ay = A for the 
reliability of the movement vectors is given by: 

N 



{( ° (x = (y} [* = *x = *y] being the weighting factor of an image block or the movement 
vector thereof. 

The [present] invention can be used, for example, to compensate a 
movement of a moving camera or { 

}also for the movement compensation of a camera which is integrated in a mobile 
communication unit {(vid e o mobil e phon e ).} [, such as a video mobile phone.] 

{Th e i nv e ntion can i n add i t i on be us e d for i mage s e gm e ntation as d e scr i b e d 
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Th e inv e ntion i s to b e s ee n vivid l y i n that} [According to the present invention,] 

nriovement vectors which are determined { i n any cas e } during the block-based 
image encoding fafe}[, can be] used to determine a global movement between a 
camera and an image sequence taken by the camera. 

However, during determination of the movement account is taken only of movement 
vectors which are assigned to image blocks which are situated in a prescribed 
region. { 

}The movement vectors of the image blocks are weighted in accordance with their 
reliability for the purpose of calculating the global movement. 
{Th e fo ll ow i ng pub li cat i ons ar e c i t e d in this docum e nt: 

[1] R. M e ch, M. Wo ll born, A Nois e Robust M e thod for 2D Shap e Estimat i on of 
Moving Obj e cts i n V i d e o S e qu e nc e s Cons i d e ring a Mov i ng Cam e ra, Workshop on 
I mag e Ana l ys i s for Mu l t i m e d i a I nt e ractiv e S e rv i c e s, B el gium, Jun e 1997 

[2] S. Co l onn e s e e t a l ., Adapt i v e S e gm e ntat i on of Mov i ng Obj e ct v e rsus Background 
for Vid e o Encoding, Proc ee d i ngs of SPIE Annua l Symposium, Vo l . 3164, San D ie go, 
August 1997 

[3] S.S. B e auch e m i n, J.L. Barron, Th e Computation of Optica l f l ow, ACM 
Comput i ng Surv e ys, Vo l . 27, No, 3, pag e s 366- 433, S e pt e mber 19 9 5 

[4] M. B le ri i n, Displac e m e nt Est i mation by I l i e rarch i ca l B l ockmatching, SP I E, Vo l . 
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1001, Visua l Commun i cations and I mag e Proc e ssing '80, pag e s 9 42 - 9 51, 1 9 00 

[5] I TU - T, Int e mationa l T ele communicat i on Un i on, T e l e communications S e ctor of 
I TU, Draft I TU-T R e comm e ndation H .2 6 3, V i d e o e ncod i ng for l ow bitrat e 
communicat i on, 2nd May 1996 

Pat e nt Cla i ms 

1 . A m e thod for comput e r-aid e d d e t e rminat i on of a mov e m e nt which und e r l i e s a 
d i git i z e d i mag e , 

- in wh i ch th e dig i tiz e d i mag e contains pix el s wh i ch ar e group e d i nto i mag e blocks, 
" i n which a mov e m e nt e st i mation is carr ie d out for e ach i mag e block, as a r e su l t of 
wh i ch a mov e m e nt v e ctor i s d e t e rm i n e d for e ach i mag e b l ock, which mov e m e nt 
v e ctor i s assign e d to th e r e sp e ct i v e i mag e b l ock, 

" in wh i ch mov e m e nt v e ctors ar e s e l e ct e d which ar e assign e d to an imag e block 
which i s situat e d i n a pr e scr i b e d r e g i on of th e dig i t i z e d i mag e , 
" in which param e t e rs of a mov e m e nt mod el ar e d e t e rmin e d from th e s ele ct e d 
mov e m e nt v e ctors, and 

" i n wh i ch th e mov e m e nt of th e d i g i t i z e d imag e i s d e scr i b e d by th e d e t e rm i n e d 
mov e m e nt mod el . 

2. Th e m e thod as c l aim e d i n c l a i m 1 , in wh i ch th e pr e scribed r e gion is form e d by 
i mage blocks wh i ch ar e situat e d at a prescr i b e d f i rst d i stanc e from an e dg e of th e 
d i gitiz e d imag e . 
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3. Th e m e thod as c l a i m e d i n c l a i m 2, i n wh i ch th e pr e scrib e d r e g i on is form e d by 
i mag e blocks wh i ch ar e s i tuat e d at a pr e scrib e d s e cond d i stanc e from th e midd le of 
th e dig i t i z e d i mag e , 

4. Th e m e thod as c l aim e d in on e of c l aims 1 to 3, i n wh i ch th e pr e scr i b e d r e g i on is 
var ie d i n an it e rat i v e m e thod. 

5. Th e m e thod as claim e d in on e of claims 1 to 4, i n which th e mov e m e nt e stimation 
i s p e rform e d by a blockwis e compar i son of th e imag e b l ock in th e d i git i z e d imag e 
w i th an i mag e b l ock i n a t e mporal l y pr e c e ding i mag e wh i ch, insid e a s e arch spac e of 
pr e scr i b e d shap e and s i z e , i s 

d i sp l ac e d by a pr e scr i b e d va l u e r el ativ e to th e i mag e b l ock in th e d i g i t i z e d imag e . 

6. The m e thod as c l a i m e d i n on e of c l a i ms 1 to 5, i n which th e d e t e rmined 
mov e m e nt i s comp e nsat e d. 

7. Th e m e thod as c l aim e d in cla i m 6, us e d i n a mobil e arrang e m e nt whos e 
mov e m e nt i s comp e nsat e d w i th th e aid of th e m e thod. 

8. The m e thod as c l aim e d i n c l aim 7, in which th e arrang e m e nt i s a cam e ra. 

9. The m e thod as c l aim e d i n c l a i m 8, i n wh i ch th e arrang e m e nt i s a cam e ra which i s 
i nt e grat e d in a mob ile communication d e v i c e . 
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10. An arrang e m e nt for d e t e rmin i ng a mom e nt wh i ch und e rl ie s a d i g i t i z e d i mag e , 
having a proc e ssor wh i ch is s e t up i n such a way that th e fol l ow i ng st e ps can b e 
carr ie d out: 

" th e d i g i t i z e d i mag e conta i ns pix e ls wh i ch ar e group e d i nto imag e b l ock s , 
" a mov e m e nt e stimat i on i s carr ie d out for e ach i mag e b l ock, as a r e su l t of wh i ch a 
mov e m e nt v e ctor is d e t e rm i ned for e ach imag e b l ock, wh i ch mov e m e nt v e ctor is 
ass i gn e d to the r e spect i v e i mag e b l ock, 

" mov e m e nt v e ctors ar e s ele ct e d which ar e ass i gn e d to an imag e block which i s 
s i tuat e d i n a pr e scrib e d r e g i on of th e d i gitiz e d i mag e , 

■ param e t e rs of a mov e m e nt mod el ar e d e t e rm i n e d from the s ele ct e d mov e m e nt 
v e ctors, and 

" th e mov e m e nt of th e d i g i t i z e d i mag e is d e scr i b e d by th e d e t e rm i n e d mov e m e nt 
modefr 

1 1 . Th e arrang e m e nt as cla i m e d in cla i m 10, 

in wh i ch th e proc e ssor i s s e t up in such a way that th e pr e scrib e d r e g i on i s form e d 
by i mag e blocks wh i ch ar e situat e d at a pr e scr i b e d f i rst d i stanc e from an e dg e of th e 
digit i z e d imag e . 

12. Th e arrang e m e nt as c l aim e d i n Cla i m 1 1 , i n which th e proc e ssor i s s e t up in 
such a way that th e pr e scr i b e d r e g i on i s form e d by i mag e b l ocks which ar e situat e d 
at a pr e scr i b e d s e cond d i stanc e from th e midd le of th e digit i z e d i mag e . 

13. Th e arrang e m e nt as c l aimed in on e of c l a i ms 10 to 12, i n which th e processor i s 
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s e t up in such a way that th e pr e scr i b e d r e g i on i s var ie d i n an i t e rativ e m e thod. 

14. The arrang e nn e nt as c l a i m e d i n on e of c l a i ms 10 to 13 Jn wh i ch th e proc es sor is 
s e t up i n such a way that th e mov e m e nt e st i mat i on i s p e rform e d by a blockw i s e 
comparison of th e i mag e b l ock i n th e dig i t i z e d i mag e w i th an i mag e block in a 
t e mporally pr e c e d i ng i mag e wh i ch, ins i d e a s e arch spac e of pr e scr i b e d shap e and 
siz e , is d i splac e d by a pr e scr i b e d va l u e r e lat i v e to th e imag e b l ock i n th e dig i tiz e d 
i mag e . 

15. Th e arrang e m e nt as cla i m e d i n on e of cla i ms 10 to 14, i n which th e proc e ssor i s 
s e t up i n such a way that th e d e t e rmin e d mov e m e nt i s comp e nsat e d. 

16. Th e arrang e m e nt as cla i m e d i n c l aim 15, us e d in a mob i l e d e vic e . 

17. Th e arrang e m e nt as claim e d in c l aim 1 6 , us e d i n a cam e ra. 

18. Th e arrangem e nt as claim e d i n c l aim 17, us e d i n a commun i cation unit w i th a 
camera. 

Abstract 

M e thod and arrangem e nt for d e t e rm i n i ng a mov e m e nt wh i ch und e r lie s a d i g i t i z e d 
image 

Th e i mag e contains pix el s which ar e group e d i nto imag e b l ocks. A mov e m e nt 
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e st i mat i on i s carri e d out for e ach imag e b l ock (st e ps 502, 503, 504, 505). Th e 
mov e ment v e ctors d e t e rm i n e d i n th i s cas e ar e s el ect e d wh e n th e y ar e ass i gn e d to 
an i mag e block wh i ch i s s i tuat e d i n a pr e scrib e d r e gion of th e d i git i z e d i mag e (st e p 
506). Param e t e rs of a mov e m e nt mod el ar e d e t e rm i n e d (st e p 507) from th e s e l e ct e d 
mov e m e nt v e ctors and th e mov e m e nt of th e dig i tiz e d i mag e is d e scr i b e d by th e 
d e t e rmin e d mov e m e nt mod e l. 

figur e 5} [Zooming and rotating of the video camera can be specified only 
unreliably by movement vectors which are assigned to image blocks which are 
grouped in a region around the middle of the image. In this case, the 
prescribed region clearly forms a "mask" in the form of a "perforated" 
rectangle inside the digitized image. Iterations are introduced to determine the 
movement model by modifying the "mask" after determining the parameters of 
the movement model. The modified "mask" is used to recalculate the 
parameters of the movement model. The "mask" can be modified by virtue of 
the fact that blocks whose movement vectors deviate from those of the 
movement model, and whose deviation exceeds a threshold value with 
reference to a prescribable distance measure, are eliminated from the 
prescribed region. The prescribed region is formed by image blocks whose 
movement can be estimated reliably, based upon an associated prediction 
error which is below a prescribed threshold, or the variance of the prediction 
error in the search zone is above a threshold. A "weighting mask" is used 
instead of the ^binary mask" such that blocks or their movement vectors are 
weighted with factors. These can be different for the X-<:omponent and 
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Y-component of the movement vector. The weightings feature in the 
calculation of the parameters of the movement model, and the determined 
movement can be used to compensate an actual movement of the 
arrangement with the aid of which an image is taken. 

Although preferred embodiments of the present invention have been 
described herein, it is to be understood that the invention is not limited to 
these embodiments and that various changes and modifications thereto may 
be made by persons having skill in the art to which the invention pertains, 
without departing from the scope or spirit of the invention, which is defined by 
the following claims. - -] 
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